Systems and Methods for Automatic Vertebra Edge Detection, Segmentation and Identification in 3D Imaging

ABSTRACT

Systems and methods for automatic accurate and efficient segmentation and identification of one or more vertebra in digital medical images using a coarse-to-fine segmentation.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a utility application, which claims thebenefit of U.S. Provisional Application No. 61/241,221, filed Sep. 10,2009, which is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to computer-aided diagnosis systems andmethods and, more specifically, to improved systems and methods fordetection, segmentation, and identification of vertebrae inthree-dimensional imaging.

BACKGROUND

The field of medical imaging has seen significant advances since thetime X-Rays were first used to determine anatomical abnormalities.Medical imaging hardware has progressed in the form of newer machinessuch as Medical Resonance Imaging (MRI) scanners, Computed AxialTomography (CAT) scanners, etc. Because of large amount of image datagenerated by such modern medical scanners, there has been and remains aneed for developing image processing techniques that can automate someor all of the processes to determine the presence of anatomicalabnormalities in scanned medical images.

Recognizing anatomical structures within digitized medical imagespresents multiple challenges. For example, a first concern relates tothe accuracy of recognition of anatomical structures within an image. Asecond area of concern is the speed of recognition. Because medicalimages are an aid for a doctor to diagnose a disease or condition, thespeed with which an image can be processed and structures within thatimage recognized can be of the utmost importance to the doctor reachingan early diagnosis. Hence, there is a need for improving recognitiontechniques that provide accurate and fast recognition of anatomicalstructures and possible abnormalities in medical images.

Digital medical images are constructed using raw image data obtainedfrom a scanner, for example, a CAT scanner, MRI, etc. Digital medicalimages are typically either a two-dimensional (“2-D”) image made ofpixel elements or a three-dimensional (“3-D”) image made of volumeelements (“voxels”). Such 2-D or 3-D images are processed using medicalimage recognition techniques to determine the presence of anatomicalstructures such as cysts, tumors, polyps, etc. Given the amount of imagedata generated by any given image scan; it is preferable that anautomatic technique should point out anatomical features in the selectedregions of an image to a doctor for further diagnosis of any disease orcondition.

One general method of automatic image processing employs feature basedrecognition techniques to determine the presence of anatomicalstructures in medical images. However, feature based recognitiontechniques can suffer from accuracy problems.

Automatic image processing and recognition of structures within amedical image is generally referred to as Computer-Aided Detection(CAD). A CAD system can process medical images and identify anatomicalstructures including possible abnormalities for further review. Suchpossible abnormalities are often called candidates and are considered tobe generated by the CAD system based upon the medical images.

One particular area in which the use of CAD systems would be highlyadvantageous is in the imaging of the spinal column. A precise vertebrasegmentation and identification method is in high demand due to itsimportance to, and impact on, many orthopedic, neurological, andontological applications. Unfortunately, the task of segmenting andidentifying vertebrae, even using CAD systems that are well-developedfor other anatomical structures, often proves inaccurate and thereforeinadequate. The difficulty lies in the inherent complexity of vertebrae.Each vertebra can be modeled against the same vertebra in other patientsand an average, or mean vertebra model can be created. However, thevariation within the same class of vertebra as well as the variation inneighboring structures makes vertebral modeling and imaging extremelydifficult.

Several methods have been reported addressing segmentation and/oridentification of vertebra using a wide variety of different imagingmodalities (e.g., magnetic resonance imaging (MRI), computed tomography(CT), etc.). Such prior approaches include a method to automaticallyextract and partition the spinal cord in CT images as described in Yao,J., O'Conner, S., Summers, R.: Automated Spinal Column Extraction andPartitioning. In: Proc. of IEEE ISM, pp. 390-393 (2006), which is herebyincorporated by reference herein. Another prior approach includes usinga surface-based registration for automatic lumbar vertebraidentification as described in Herring, J., Dawant, B.: Automatic LumbarVertebral Identification Using Surface-Based Registration. Computers andBiomedical Research 34(2), 629-642 (2001), which is hereby incorporatedby reference in its entirety.

More recent approaches propose a model-based solution for vertebraldetection, segmentation, and identification in CT images, as described,for example, in Klinder, T., Ostermatm, J., Ehm, M., Franz, A., Kneser,R., Lorenz, C.: Automated Model-Based Vertebra Detection,Identification, and Segmentation in CT Images. Medical Image Analysis13, 471-481 (2009), which is hereby incorporated by reference herein.The approach described in Klinder achieved competitive identificationrates of approximately 70% when identifying a single vertebra and 100%when identifying 16 or more vertebrae. However, that identificationalgorithm is based on vertebral appearance model (i.e., average volumeblock) spatial registration and matching which is extremelycomputationally consuming. In order to achieve the high identificationrates of the Klinder approach requires approximately 20-30 minutes ofcomputational time. In a real-world hospital setting, such systemdedication to a single patient's data is not practical or realistic.

Therefore there is a need for systems and methods for precisesegmentation and identification of vertebrae that is both accurate andefficient.

SUMMARY OF THE INVENTION

According to an aspect of the present disclosure, a system for automatedsegmentation of anatomical structures in image data is provided,including a digital image input for receiving at least one digital imageof an anatomical structure, a storage device, coupled to the image inputfor storing the at least one digital image, and a processor coupled tothe storage device. The processor being configured to receive the atleast one digital image, analyze the at least one digital image andautomatically locate at least one anatomical landmark, select a templatefrom a database of pre-defined templates based upon the at least oneanatomical landmark, and segment the at least one digital image byapplying a coarse segmentation and a fine segmentation to obtain anoptimized segmented template. Applying the coarse segmentationcomprises, for each of a plurality of sub-regions of the anatomicalstructure, applying a corresponding sub-region of the selected templateto the anatomical structure, calculating a first edge response,deforming the sub-region of the selected template, calculating adeformation edge response, selecting the sub-region or deformedsub-region of the template having a highest edge response, and combiningthe highest edge-response sub-region from each of the plurality ofsub-regions into a single coarse-segmented template. Applying the finesegmentation comprises, for each of a plurality of patches of thecoarse-segmented template, calculating an edge-response of the patchrelative to the anatomical structure, deforming the patch, calculatingan edge-response of the deformed patch, selecting the patch or deformedpatch having a highest edge-response, and combining the highestedge-response patches into a single optimized segmented template. Thedigital image can be at a three-dimensional (3D) image, and the templatecan be a 3D surface template. Receiving the digital image can includeacquiring the image using an imager selected from: computer tomography(CT), positron emission tomography (PET), single photon emissioncomputed tomography (SPECT), magnetic resonance (MR), and ultrasound.Selecting a template can include selecting a template from a database ofpre-defined templates based upon the at least one anatomical landmark,performing a plurality of rigid transformation on the selected template,calculating a fit response for each transformed template, and selectinga transformed template having a highest fit response. Deforming thetemplate can include performing a plurality of translations of thetemplate along each of the x, y, and z axes, performing a plurality ofre-scalings of the template along each of the x, y, and z axes, andperforming a plurality of rotations of the template along each of the x,y, and z axes. Subsequent to combining the highest edge-responsesub-region from each of the plurality of sub-regions, the processor canperform a smoothing operation to smooth the single coarse-segmentedtemplate. Deforming the patch can include calculating a mean normal forthe patch, and deforming the patch along the mean normal. Subsequent tocombining the highest edge-response patches, the processor can perform asmoothing operation to smooth the single optimized segmented template.

According to another aspect of the present disclosure, a method forautomated segmentation of anatomical structures in image data isprovided, including acquiring at least one medical image of ananatomical structure, automatically locating at least one anatomicallandmark in the at least one medical image, selecting a template from adatabase of pre-defined templates based at least in part on the at leastone anatomical landmark, and segmenting the at least one medical imageby applying a coarse segmentation and a fine segmentation to obtain anoptimized segmented template. Applying the coarse segmentation caninclude, for each of a plurality of sub-regions of the anatomicalstructure, applying a corresponding sub-region of the selected templateto the image of the anatomical structure, calculating a firstedge-response, deforming the sub-region of the selected template,calculating a deformation edge-response, selecting the sub-region ordeformed sub-region of the template having a highest edge-response, andcombining the highest edge response sub-region from each of theplurality of sub-regions into a single coarse-segmented template.Applying a fine segmentation can include, for each of a plurality ofpatches of the coarse-segmented template, calculating an edge-responseof the patch relative to the anatomical structure, deforming the patch,calculating an edge-response of the deformed patch, selecting the patchor deformed patch having a highest edge-response, and combining thehighest edge-response patches into a single optimized segmentedtemplate. The digital image can be at least one three-dimensional (3D)image. Acquiring the digital image can include acquiring the image usingan imaging technique selected from: computer tomography (CT), positronemission tomography (PET), single photon emission computed tomography(SPECT), magnetic resonance (MR), and ultrasound. Selecting a templatecan include selecting a template from a database of pre-definedtemplates based at least in part on the at least one anatomicallandmark, performing a plurality of rigid transformation on the selectedtemplate, calculating a fit response for each transformed template, andselecting a transformed template having a highest fit response.Deforming the template can include performing a plurality oftranslations of the template along each of the x, y, and z axes,performing a plurality of re-scalings of the template along each of thex, y, and z axes, and performing a plurality of rotations of thetemplate along each of the x, y, and z axes. Subsequent to combining thehighest edge-response sub-region from each of the plurality ofsub-regions, a smoothing operation can be performed to smooth the singlecoarse-segmented template. Deforming the patch can include calculating amean normal for the patch, and deforming the patch along the meannormal. Subsequent to combining the highest edge-response patches, asmoothing operation can be performed to smooth the single optimizedsegmented template.

According to yet another aspect of the present disclosure, a system forautomated identification of vertebrae in image data is provided,including a digital image input for receiving at least one digital imageof at least one vertebra, a storage device, coupled to the image inputfor storing the at least one digital image, and a processor coupled tothe storage device. The processor being configured to receive the atleast one digital image, analyze the at least one digital image andautomatically locate a plurality of anatomical landmarks, calculate acorrespondence for each vertebra in the digital image based at least inpart upon the plurality of anatomical landmarks, and relative to each ofa plurality of pre-defined mean vertebra models, and identify eachvertebra in the digital image based on a highest correspondence to apre-defined mean vertebra model. The vertebra can be a thoracicvertebra, and the plurality of pre-defined mean vertebra models can betwelve thoracic vertebra models.

According to another aspect of the present disclosure, a method forautomated identification of vertebrae in image data is provided,including receiving at least one digital image of at least one vertebra,analyzing the at least one digital image and automatically locate aplurality of anatomical landmarks, calculating a correspondence for eachvertebra in the digital image based at least in part upon the pluralityof anatomical landmarks, and relative to each of a plurality ofpre-defined mean vertebra models, and identifying each vertebra in thedigital image based on a highest correspondence to a pre-defined meanvertebra model. The at least one vertebra can be a thoracic vertebra,and the plurality of pre-defined mean vertebra models can be twelvethoracic vertebra models.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the present disclosure and many of theattendant aspects thereof will be readily obtained as the same becomesbetter understood by reference to the following detailed descriptionwhen considered in connection with the accompanying drawings.

FIG. 1 is a diagram illustrating the feature sampling and calculatingaccording to exemplary embodiments of the present disclosure.

FIGS. 2A-C illustrate the application of surface templates to imagedvertebrae in accordance with aspects of the present disclosure.

FIG. 3 illustrates exemplary sub-regions of a vertebra surface inaccordance with an aspect of the present disclosure.

FIGS. 4A and 4B illustrate application of a coarse segmentation to avertebra image in accordance with an aspect of the present disclosure.

FIGS. 4C and 4D illustrate application of a fine segmentation to avertebra image in accordance with an aspect of the present disclosure.

FIG. 5 illustrates exemplary segmentation results in axial, sagittal andcoronal views in accordance with an aspect of the present disclosure.

FIGS. 6A and 6B show successful vertebra identification rates whenidentifying a single vertebra and a string of vertebrae, respectively,in accordance with an aspect of the present disclosure.

FIG. 7 shows an example of a computer system capable of implementing themethod and system according to embodiments of the present disclosure

FIG. 8 is a flow chart illustrating a method for vertebra segmentationaccording to an exemplary embodiment of the present disclosure.

FIG. 9 is a flow chart illustrating a method for vertebra identificationaccording to an exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, numerous specific details are set forthsuch as examples of specific components, devices, methods, etc., inorder to provide a thorough understanding of embodiments of the presentinvention. It will be apparent, however, to one skilled in the art thatthese specific details need not be employed to practice embodiments ofthe present invention. In other instances, well-known materials ormethods have not been described in detail in order to avoidunnecessarily obscuring embodiments of the present invention. While theinvention is susceptible to various modifications and alternative forms,specific embodiments thereof are shown by way of example in the drawingsand will herein be described in detail. It should be understood,however, that there is no intent to limit the invention to theparticular forms disclosed, but on the contrary, the invention is tocover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the invention.

The term “x-ray image” as used herein may mean a visible x-ray image(e.g., displayed on a video screen) or a digital representation of anx-ray image (e.g., a file corresponding to the pixel output of an x-raydetector). The term “in-treatment x-ray image” as used herein may referto images captured at any point in time during a treatment deliveryphase of a radiosurgery or radiotherapy procedure, which may includetimes when the radiation source is either on or off. From time to time,for convenience of description, CT imaging data may be used herein as anexemplary imaging modality. It will be appreciated, however, that datafrom any type of imaging modality including but not limited to X-Rayradiographs, MRI, CT, PET (positron emission tomography), PET-CT, SPECT,SPECT-CT, MR-PET, 3D ultrasound images or the like may also be used invarious embodiments of the invention.

Unless stated otherwise as apparent from the following discussion, itwill be appreciated that terms such as “segmenting,” “generating,”“registering,” “determining,” “aligning,” “positioning,” “processing,”“computing,” “selecting,” “estimating,” “detecting,” “tracking” or thelike may refer to the actions and processes of a computer system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (e.g., electronic) quantities within thecomputer system's registers and memories into other data similarlyrepresented as physical quantities within the computer system memoriesor registers or other such information storage, transmission or displaydevices. Embodiments of the methods described herein may be implementedusing computer software. If written in a programming language conformingto a recognized standard, sequences of instructions designed toimplement the methods can be compiled for execution on a variety ofhardware platforms and for interface to a variety of operating systems.In addition, embodiments of the present invention are not described withreference to any particular programming language. It will be appreciatedthat a variety of programming languages may be used to implementembodiments of the present invention.

As used herein, the term “image” refers to multi-dimensional datacomposed of discrete image elements (e.g., pixels for 2-D images andvoxels for 3-D images). The image may be, for example, a medical imageof a subject collected by computer tomography, magnetic resonanceimaging, ultrasound, or any other medical imaging system known to one ofskill in the art. The image may also be provided from non-medicalcontexts, such as, for example, remote sensing systems, electronmicroscopy, etc. Although an image can be thought of as a function fromR³ to R or R⁷, the methods of the inventions are not limited to suchimages, and can be applied to images of any dimension, e.g., a 2-Dpicture or a 3-D volume. For a 2- or 3-dimensional image, the domain ofthe image is typically a 2- or 3-dimensional rectangular array, whereineach pixel or voxel can be addressed with reference to a set of 2 or 3mutually orthogonal axes. The terms “digital” and “digitized” as usedherein will refer to images or volumes, as appropriate, in a digital ordigitized format acquired via a digital acquisition system or viaconversion from an analog image.

Exemplary embodiments of the present invention seek to provide anapproach for automatically segmenting and/or identifying one or morevertebra within a digital medical image. The following descriptionrefers primarily to thoracic vertebrae, which are readily visible indigital images during lung examinations. However, the systems andmethods of the present disclosure can be readily extended and applicableto cervical and lumbar vertebrae.

In order to accomplish the accurate and efficient automatic segmentationand identification of vertebrae in the present disclosure, alearning-based bone structure edge detection algorithm is employedincluding efficient and effective gradient steerable features and robusttraining data sampling. Additionally, a hierarchical coarse-to-finedeformable surface based segmentation method is provided based at leastin part on the response maps from the learned edge detector. Anefficient vertebra identification method and system is described usingmean vertebral shapes to identify imaged vertebrae.

Due to complex neighboring structures around vertebrae as well asimaging noise, common edge detectors such as a Canny operator, oftenproduce detected edges that are leaking and spurious. In accordance withaspects of the present disclosure, a learning-based object specific edgedetection algorithm is provided to achieve robust edge detection. Thisalgorithm is similar to semantic object-level boundary lineation innatural image, as described for example in Dollar, P., Tu, Z., Belongie,S.: Supervised Learning of Edges and Object Boundaries. In: CVPR (2006),which is incorporated by reference herein in its entirety.

The learning-based edge detection of the present disclosure wasdeveloped by supervised bone edge detection. Twelve thoracic vertebraewere manually segmented from twenty CT volumes for training.Corresponding triangulated surfaces were generated using a Marching Cubealgorithm, with approximately 10,000 triangular faces per vertebramodel. Along the normal direction of the vertebral bone boundary,intensity values roughly formed a ridge pattern. The edge detection ofthe present disclosure is designed to describe the characteristics ofboundary appearance, which allows for statistical training.

As shown in FIG. 1, for each triangle face of the surface mesh, fivepoints are sampled (called a sampling parcel) along the face normaldirection at one voxel intervals. Specifically, given a point x on thenormal line and n the normal vector, the sampling parcel associated withx is:

P(x)={(x−2n,x−n,x+n,x+2n}

For each of the five sampling points three features are computed:Intensity I, projections of gradient onto the normal direction ∇1I·n,∇2I·n, where ∇1I and ∇2I are gradient vectors computed using aderivative of Gaussian with two different kernel scales. In total, thefeature vector of a point x, denoted by F(x) has fifteen elements:

F(x)={I(y),∇1I(y)·n,∇2I(y)·n/yεP(x)}

FIG. 1 illustrates the sampling parcel and its associated features. Thesteerable features are oriented-gradient pattern descriptor withsimplified computation.

The vertebra edge detector is trained by training positive and negativeboundary voxels obtained from manually segmented vertebra meshes asdetailed below. For a triangle face center c, the boundary parcel isdefined as

P(c)=(c−2n,c−n,c,c+n,c+2n)

and the interior parcel is defined as

P(c−3n)={c−5n,c−4n,c−3n,c−2n,c−n}

and the exterior parcel is defined as

P(c+3n)={c+n,c+2n,c+3n,c+4n,c+5n}

That is, the interior parcel is three voxels away backward from theboundary parcel while the exterior parcel is three voxels forward, wherethree is adjustable. The corresponding feature vectors F(c), F(c−3n),F(c+3n) can also be computed. F(c) is labeled as positive class (i.e.,boundary), and both F(c−3n) and F(c+3n) are labeled as negative class(i.e., non-boundary) as illustrated in FIGS. 2A and 2B. Thus, eachtriangle face provides one positive data point and two negative datapoints. Given one vertebra surface mesh with approximately 10,000 faces,sufficient and adequate training feature vectors are obtained. Note thata single and unified bony edge detector will be learned for each of thevertebrae, described throughout as the twelve thoracic vertebrae.Implicit, object “inside-outside” learning is described in, for example,Zhan, Y., Shen, D.: Deformable Segmentation of 3D Ultrasound ProstateImages Using Statistical Texture Matching Method. IEEE Trans. on MedicalImaging (2006), which is incorporated herein by reference in itsentirety. Compared with such implicit, object “inside-outside” learning,the boundary/non-boundary delineation strategy described herein directlyfocuses on modeling the runtime boundary localization process (i.e.,explicitly moving towards classified boundary positives), and delivershigher precision. Additionally, in such “inside-outside” methods, theboundary has to be further inferred from the transition of (object)internal positives and external negatives which is not a trivialconsideration.

The feature vectors depend on the normal direction of triangle faces sothat the edge detector is sensitive to the initialization of the surfacetemplate. According to the systems and methods of the presentdisclosure, a selected surface template or model is first roughlyregistered with an image or set of images by automatic detection ormanual alignment, thus the normal direction of the surface template ormodel can not perfectly coincide with the true bony normal. To make thedetector more robust to mis-alignment errors and thereby make the laterdeformable model convergent, some “noisy” training samples aresynthesized by stress testing. Particularly, some random disturbanceswere added to the orientations and scales of the template model so thatthe template surface model does not accurately overlap with the manualsegmentation. Considering a similarity transform, a random numberbetween 0.9 and 1.1 for each of the three scales, and a random anglebetween −(π/10) and (π/10) for each of the three orientation angles areused. The true boundary parcels, as well as interior and exteriorparcels are defined using ground truth positions but with disturbedtemplate surface normals as shown in FIG. 2B. The corresponding featurevectors are consequently calculated (with the disturbed face normals)and added into the training sets. The random disturbance process isrepeated 10 times for each training mesh to guarantee enough noisysamples. A linear or quadratic discriminant (LDA, QDA) classifier istrained based on the combined non-disturbed and disturbed featurevectors. Both LDA and QDA were evaluated and it was determined that LDAyields more robust results. Experiment results are computed using LDA.Finally, given a voxel x and its feature vector F(x), the classifierwill assign a value L(x)ε[0, 1.0] which indicates the likelihood of xbeing a boundary point.

In accordance with aspects of the present disclosure, in order toaccurately and efficiently segment a digital image of one or morevetebra, an image or a series of images of an anatomical structure, inthis case a vertebra are acquired. A system automatically locates atleast one and preferably at least three anatomical landmarks within theimage and selects a template from a database of pre-defined templatesbased at least in part on the identified anatomical landmarks. Thesystem can optionally apply a series of rigid transformations to theselected template and continually calculate response values in order tobetter fit the template to the anatomical structure in the image. Inaccordance with the present disclosure, the system then applies a coarsesegmentation to the image followed by a fine segmentation. After eachsegmentation step, the system can run a smoothing operation such as aGaussian smoothing in order to smooth out any disturbances caused by thecoarse or fine segmentation operations.

The main idea of segmentation is to deform the surface template meshtowards boundary points detected by the learned edge detector. After thesurface template is initially positioned into a new volume, (thetemplate can be initialized using strategies similar to marginal spacelearning) the edge detector calculates the edge likelihoods L(x) forvoxels along the normal directions of all mesh faces, where a responsemap can be generated. As shown in FIG. 2C the response map isinformative but unavoidably noisy. To guarantee the surface shapetopology and smoothness during deformation/segmentation, the presentdisclosure employs a hierarchical deformation scheme of first performinga coarse deformation of sub-regions of the vertebra, then performing afine segmentation of a multitude of patches. Patches are referred toindicate that points in the same neighborhood move together during finesegmentation.

As shown in FIG. 2A, a surface template 202 is perfectly aligned withthe true boundary of the vertebra. FIG. 2B illustrates a surfacetemplate 204 that is mis-aligned. Plus and minus signs in FIG. 2Billustrate positive and negative samples, respectively. FIG. 2Cillustrates a response map of vertebra edge detection in the sectionview of a 3D CT volume. Template surface 206 is shown in partialalignment with the voxels identified as boundary points, shown as line208.

Each vertebra is divided into 12 sub-regions, as shown in FIG. 3. Inorder to maintain the shape of these sub-regions, a similaritytransformation to each sub-region is applied such that the totalresponse of edge detection is maximum in the transformed configuration.For a sub-region S and some face center f on it, systems and methods inaccordance with the present disclosure calculate a similaritytransformation T satisfying

$\hat{T} = {\arg {\max\limits_{T \in T}{\sum\limits_{f \in S}{\mathcal{L}\left( {T(f)} \right)}}}}$

where T is the set of similarity transformations T. Searching theoptimal T involves the 9-dimensional parameters of (Tx, Ty, Tz, Sx, Sy,Sz, θx, θy, θz). In order to perform an exhaustive search with five gridsteps for each parameter would require a possible 5⁹ transformationswhich is computationally infeasible. Therefore, systems and methods inaccordance with the present disclosure reduce the search space byperforming a three-stage search. First a search for (Tx, Ty, Tz) iscompleted with displacement {−4, −2, 0, 2, 4} voxels for eachtranslation. Second, with fixed (Tx, Ty, Tz), the system searches for(Sx, Sy, Sz) with discretization grids of {0.8, 0.9, 1.0, 1.1, 1.2} foreach scaling. Third, with fixed optimal translation and scaling, thesystem searches for (θx, θy, θz) with intervals of {(−π/10), (−π/20), 0,(π/20), (π/10)} for each orientation. In this way, the system only needsto consider 5³*3 or 375 possible poses and select the one with thestrongest response as T. This heuristic searching strategy turns out tobe effective in capturing the true pose of sub-regions though it mightbe sub-optimal because the optimal search is computationallyunrealistic. FIG. 4A illustrates the coarse segmentation search processgraphically. Template sub-region 402 is deformed to deformed sub-regiontemplates 404, including optimal deformed sub-region template 404′.

After the optimal similarity transformation is found for eachsub-region, a smooth deformation of the whole surface can be obtainedusing simple Gaussian smoothing. For example, let S₁, S₂, S₁₂ denote thetwelve sub-regions of a vertebra, and T₁, T₂, . . . , T₁₂ be thecorresponding optimal transform. Denote an arbitrary vertex v in thetemplate surface and u a vertex in a certain sub-region. Then the newposition of v is

$v^{\prime} = {v + {\lambda {\sum\limits_{i = 1}^{12}{\sum\limits_{w \in S_{i}}{\left( {{T_{i}(w)} - w} \right){K\left( {w - v} \right)}}}}}}$where ${K(x)} = ^{- \frac{x^{2}}{2\sigma^{2}}}$

is the Gaussian kernel and λ is a regulation parameter. FIG. 4Billustrates the final result of the coarse segmentation or “deformationof sub-region” stage of the method of the present disclosure. Thesurface mesh 406 is more closely aligned with the true boundary througharticulated similarity maneuvers, although in several areas, the surfacemesh 406 still has a certain distance from the true boundary. Thisdistance will be corrected by the fine segmentation step that follows.

After the coarse segmentation, or deformation of sub-regions, thesurface mesh or surface template is in approximately overlappingposition with the vertebra's boundary in the exemplary CT volume asshown in FIG. 4B. Next, according to the methods of the presentdisclosure, a fine segmentation is performed. Unlike the coarsesegmentation that divides the surface mesh or template into twelvesub-regions, the fine segmentation divides the template into 200 patchesor local neighborhoods for each vertebra mesh surface. Each patchcontains approximately 50 mesh faces. For each patch (denoted as PT),its mean normal is computed as:

$\overset{\_}{n} = {\frac{1}{N}{\sum\limits_{f \in {PT}}{n(f)}}}$

where f is a face in the patch and n(f) is the unit normal of the face.Then the patch is moved along its mean normal direction in search of thestrongest or highest response. That is, systems and methods inaccordance with the present disclosure seek to optimize this term:

$\hat{i} = {\arg {\max\limits_{i}{\sum\limits_{f \in S}{\mathcal{L}\left( {f + {i\overset{\_}{n}}} \right)}}}}$

where the search range is limited in i=−6, −5, . . . , 5, 6. FIG. 4Cillustrates graphically how a single patch 408 (enlarged for purposes ofillustration) is displaced along its mean normal direction in search ofthe true boundary. After all patches have been deformed and optimized attheir optimal displacement, the system can perform a smooth deformationagain by Gaussian smoothing. FIG. 4D illustrates the segmentation resultfollowing both the coarse and fine segmentation steps as well assmoothing operations after each of the segmentation steps.

As is visible in FIG. 4D, the surface mesh or template accuratelycaptures the true boundary of the vertebra. The two-stage,coarse-to-fine deformation of surface model template guarantees theaccuracy of segmentation as well as the smoothness of the shapes, usingarticulated similarity transforms and non-rigid transforms respectively.

In order to identify an imaged vertebra automatically, the abovedescribed segmentation algorithm was applied for training purposes to 40volumes at 1 mm by 1 mm by 1 mm resolution and approximately 15-20surface meshes were obtained per thoracic vertebra. Vertexcorrespondence across meshes for each vertebra is also directlyavailable since surface meshes are deformed by the same template.Therefore, the system of the present disclosure can compute the meanvertebra shapes by taking the arithmetical mean of correspondingvertices' positions. There are 12 thoracic vertebrae, namely T1, T2, . .. , T12. Vertebra identification is to label a segmented vertebra to beone of the twelve. In this context, given a single vertebra subvolume, asystem in accordance with the present disclosure carries out theidentification process by testing which mean shape has the maximumresponse to an imaged vertebra. The twelve mean vertebral shape modelsare applied to an imaged vertebra one after the other in order tocalculate the supervised edge response scores without any deformation.The mean shape with the strongest response is assigned as the label ofthat vertebra.

Let M₁, M₂, . . . , M₁₂ denote the twelve mean shapes and f is anarbitrary face center in the mean shapes. One way to calculate theresponses is to compute the overall likelihood of boundary

$\hat{i} = {\arg {\max\limits_{i}{\sum\limits_{f \in M_{i}}{\mathcal{L}(f)}}}}$

Another way is to count the number of faces with high probability to beboundary points,

$\hat{i} = {\arg {\max\limits_{i}{\sum\limits_{f \in M_{i}}1_{{\mathcal{L}{(f)}} > \alpha}}}}$

were α is a threshold. The present disclosure allows for either of theabove calculation methods, but the second method proves more robustagainst outliers and imaging noise by tolerating up to (1−α) portion ofdata being polluted or not at the correct spatial configuration, andtake α=0.8 which is used for the experimental results detailed below.

Additionally, the identification method described above can be extendedto multiple vertebrae, i.e., a vertebra string. By using more vertebraand therefore more information, multiple vertebral identificationfurther increases the accuracy of identification.

Results

In experimental testing, the above-described automatic segmentationalgorithm was applied to 40 volumes of thoracic scans and evaluatedusing four-fold cross validation. In implementation, the sub-regiondeformation step (coarse segmentation) was run multiple (m) timesfollowed by patch-based deformation (fine to segmentation) being run ntimes, where m and n are empirically optimized to be 3 and 4,respectively. Supervised edge detection is performed in each iterationto reflect the runtime vertebra mesh surface configuration. FIG. 5 showsexamples of segmentation in axial, sagittal, or coronal view, for visualinspection. To quantitatively evaluate the segmentation algorithm, thedistance of a vertex on the fitted mesh to the closest mesh point (notnecessarily a vertex) of the ground truth is used. The ground truth meshis generated by manual segmentation for experimental evaluationpurposes. The mean point-to-surface error and the standard deviation forindividual vertebra is shown in Table 1 below. Highly reliable andaccurate segmentation results were shown in the evaluation of thesystems and methods of the present disclosure, with the overall finalmean error of 0.95+/−0.91 mm.

TABLE 1 vertebra T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 mean error 1.051.11 1.03 0.93 0.99 0.92 0.83 0.75 0.89 0.79 0.94 1.21 (mm) stddeviation 0.96 0.97 1.04 1.03 1.31 0.92 0.56 0.59 0.68 0.50 0.63 1.16(mm)

For identification, the systems and methods of the present disclosurehave an average success rate of 73.1% using a single vertebra. Thissuccess rate can vary from one specific vertebra to another where therates for T5, T6, T7, and T8 are less than 60% because these fourvertebrae are so physically similar to one another. However, whenexploiting the systems and methods of the present disclosure to identifya string of vertebrae, the success rate improves significantly as thestring grows in length. With a string of 9 or more vertebrae, thesuccess rate is over 91%. The success rates for individual and stringedvertebra identification according the present disclosure are comparableor better than those achieved using prior art intensity based matching.The success rates for identification according the present disclosureare shown in FIGS. 6A and 6B.

The identification method of the present disclosure appears simpler andis in fact computationally significantly simpler, saving substantialtime and computing power. This is due to the fact that previous methodsemploy intensity based matching or volumetric mean appearance models forvertebra identification. However, it was determined that though suchmethods might appear to be more robust, the variability of neighboringstructures is so significant in patients that adjacent vertebrae can beso close to one another as to hide the boundary between them. Likewiseadjacent vertebrae may appear so far apart that the prior art systemsmisinterpret the information. The method of the present disclosuredetermines identification based solely on the clean shape model withoutsurrounding structures and therefore avoids such real-world pitfalls.

FIG. 8 illustrates an exemplary flow of steps in the method of thepresent disclosure for segmenting an image including acquiring medicalimage(s) at step 802, automatically locating anatomical landmark(s) atstep 804, selecting a template at step 806, applying a coarsesegmentation to the image(s) at step 808, and applying a finesegmentation to the image(s) at step 810.

FIG. 9 illustrates an exemplary flow of steps in the method of thepresent disclosure for identifying a vertebra including receivingimage(s) at step 902, automatically locating anatomical landmarks atstep 904, calculating correspondence for each vertebra at step 906, andidentifying each vertebra based on a highest correspondence at step 908.

System Implementations

It is to be understood that embodiments of the present invention can beimplemented in various forms of hardware, software, firmware, specialpurpose processes, or a combination thereof. In one embodiment, thepresent invention can be implemented in software as an applicationprogram tangible embodied on a computer readable program storage device.The application program can be uploaded to, and executed by, a machinecomprising any suitable architecture. The system and method of thepresent disclosure may be implemented in the form of a softwareapplication running on a computer system, for example, a mainframe,personal computer (PC), handheld computer, server, etc. The softwareapplication may be stored on a recording media locally accessible by thecomputer system and accessible via a hard wired or wireless connectionto a network, for example, a local area network, or the Internet.

FIG. 7 shows an example of a computer system which may implement amethod and system of the present disclosure. The computer systemreferred to generally as system 1000 may include, inter cilia, a centralprocessing unit (CPU) 1001, memory 1004, a printer interface 1010, adisplay unit 1011, a local area network (LAN) data transmissioncontroller 1005, a LAN interface 1006, a network controller 1003, aninternal bus 1002, and one or more input devices 1009, for example, akeyboard, mouse etc. As shown, the system 1000 may be connected to adata storage device, for example, a hard disk, 1008 via a link 1007.

The memory 1004 can include random access memory (RAM), read only memory(ROM), disk drive, tape drive, etc., or a combinations thereof. Thepresent invention can be implemented as a routine that is stored inmemory 1004 and executed by the CPU 1001. As such, the computer system1000 is a general purpose computer system that becomes a specificpurpose computer system when executing the routine of the presentinvention.

The computer system 1000 also includes an operating system and microinstruction code. The various processes and functions described hereincan either be part of the micro instruction code or part of theapplication program or routine (or combination thereof) which isexecuted via the operating system. In addition, various other peripheraldevices can be connected to the computer platform such as an additionaldata storage device and a printing device.

It is to be further understood that, because some of the constituentsystem components and method steps depicted in the accompanying figurescan be implemented in software, the actual connections between thesystems components (or the process steps) may differ depending upon themanner in which the present invention is programmed. Given the teachingsof the present invention provided herein, one of ordinary skill in therelated art will be able to contemplate these and similarimplementations or configurations of the present invention.

While the present invention has been described in detail with referenceto exemplary embodiments, those skilled in the art will appreciate thatvarious modifications and substitutions can be made thereto withoutdeparting from the spirit and scope of the invention as set forth in theappended claims. For example, elements and/or features of differentexemplary embodiments may be combined with each other and/or substitutedfor each other within the scope of this disclosure and appended claims.

What is claimed is:
 1. A system for automated segmentation of anatomicalstructures in image data comprising: a digital image input for receivingat least one digital image of an anatomical structure and; a storagedevice, coupled to the image input for storing the at least one digitalimage; and a processor coupled to the storage device and configured to:receive the at least one digital image; analyze the at least one digitalimage and automatically locate at least one anatomical landmark; selecta template from a database of pre-defined templates based upon the atleast one anatomical landmark; segment the at least one digital image byapplying a coarse segmentation and a fine segmentation to obtain anoptimized segmented template, wherein applying the coarse segmentationcomprises, for each of a plurality of sub-regions of the anatomicalstructure, applying a corresponding sub-region of the selected templateto the anatomical structure, calculating a first edge response,deforming the sub-region of the selected template, calculating adeformation edge response, selecting the sub-region or deformedsub-region of the template having a highest edge response, and combiningthe highest edge-response sub-region from each of the plurality ofsub-regions into a single coarse-segmented template, wherein applyingthe fine segmentation comprises, for each of a plurality of patches ofthe coarse-segmented template, calculating an edge-response of the patchrelative to the anatomical structure, deforming the patch, calculatingan edge-response of the deformed patch, selecting the patch or deformedpatch having a highest edge-response, and combining the highestedge-response patches into a single optimized segmented template.
 2. Thesystem of claim 1, wherein the at least one digital image comprises atleast one three-dimensional (3D) image, and the template comprises a 3Dsurface template.
 3. The system of claim 1, wherein receiving the atleast one digital image further comprises acquiring the at least onedigital image using an imager selected from: computer tomography (CT),positron emission tomography (PET), single photon emission computedtomography (SPECT), magnetic resonance (MR), and ultrasound.
 4. Thesystem of claim 1, wherein the processor being configured to select atemplate further comprises the processor being configured to: select atemplate from a database of pre-defined templates based upon the atleast one anatomical landmark; perform a plurality of rigidtransformation on the selected template; calculate a fit response foreach transformed template; and select a transformed template having ahighest fit response.
 5. The system of claim 1, wherein deforming thetemplate further comprises: performing a plurality of translations ofthe template along each of the x, y, and z axes; performing a pluralityof re-scalings of the template along each of the x, y, and z axes; andperforming a plurality of rotations of the template along each of the x,y, and z axes.
 6. The system of claim 1, wherein, subsequent tocombining the highest edge-response sub-region from each of theplurality of sub-regions, the processor is further configured to performa smoothing operation to smooth the single coarse-segmented template. 7.The system of claim 1, wherein deforming the patch further comprises:calculating a mean normal for the patch; and deforming the patch alongthe mean normal.
 8. The system of claim 1, wherein, subsequent tocombining the highest edge-response patches, the processor is furtherconfigured perform a smoothing operation to smooth the single optimizedsegmented template.
 9. A method for automated segmentation of anatomicalstructures in image data, comprising: acquiring at least one medicalimage of an anatomical structure; automatically locating at least oneanatomical landmark in the at least one medical image; selecting atemplate from a database of pre-defined templates based at least in parton the at least one anatomical landmark; segmenting the at least onemedical image by applying a coarse segmentation and a fine segmentationto obtain an optimized segmented template, wherein applying the coarsesegmentation comprises, for each of a plurality of sub-regions of theanatomical structure, applying a corresponding sub-region of theselected template to the image of the anatomical structure, calculatinga first edge-response, deforming the sub-region of the selectedtemplate, calculating a deformation edge-response, selecting thesub-region or deformed sub-region of the template having a highestedge-response, and combining the highest edge response sub-region fromeach of the plurality of sub-regions into a single coarse-segmentedtemplate, wherein applying a fine segmentation comprises, for each of aplurality of patches of the coarse-segmented template, calculating anedge-response of the patch relative to the anatomical structure,deforming the patch, calculating an edge-response of the deformed patch,selecting the patch or deformed patch having a highest edge-response,and combining the highest edge-response patches into a single optimizedsegmented template.
 10. The method of claim 9, wherein the at least onedigital image comprises at least one three-dimensional (3D) image. 11.The method of claim 9, wherein acquiring the at least one digital imagecomprises acquiring the at least one digital image using an imagingtechnique selected from: computer tomography (CT), positron emissiontomography (PET), single photon emission computed tomography (SPECT),magnetic resonance (MR), and ultrasound.
 12. The method of claim 9,wherein selecting a template further comprises: selecting a templatefrom a database of pre-defined templates based at least in part on theat least one anatomical landmark; performing a plurality of rigidtransformation on the selected template; calculating a fit response foreach transformed template; and selecting a transformed template having ahighest fit response.
 13. The method of claim 9, wherein deforming thetemplate further comprises: performing a plurality of translations ofthe template along each of the x, y, and z axes; performing a pluralityof re-scalings of the template along each of the x, y, and z axes; andperforming a plurality of rotations of the template along each of the x,y, and z axes.
 14. The method of claim 9, further comprising, subsequentto combining the highest edge-response sub-region from each of theplurality of sub-regions, performing a smoothing operation to smooth thesingle coarse-segmented template.
 15. The method of claim 9, whereindeforming the patch further comprises: calculating a mean normal for thepatch; and deforming the patch along the mean normal.
 16. The method ofclaim 9, further comprising, subsequent to combining the highestedge-response patches, performing a smoothing operation to smooth thesingle optimized segmented template.
 17. A system for automatedidentification of vertebrae in image data comprising: a digital imageinput for receiving at least one digital image of at least one vertebra;a storage device, coupled to the image input for storing the at leastone digital image; and a processor coupled to the storage device andconfigured to: receive the at least one digital image; analyze the atleast one digital image and automatically locate a plurality ofanatomical landmarks; calculate a correspondence for each vertebra inthe digital image based at least in part upon the plurality ofanatomical landmarks, and relative to each of a plurality of pre-definedmean vertebra models; identify each vertebra in the digital image basedon a highest correspondence to a pre-defined mean vertebra model. 18.The system of claim 17, wherein the at least one vertebra is a thoracicvertebra, and the plurality of pre-defined mean vertebra models aretwelve thoracic vertebra models.
 19. A method for automatedidentification of vertebrae in image data comprising: receiving at leastone digital image of at least one vertebra; analyzing the at least onedigital image and automatically locate a plurality of anatomicallandmarks; calculating a correspondence for each vertebra in the digitalimage based at least in part upon the plurality of anatomical landmarks,and relative to each of a plurality of pre-defined mean vertebra models;identifying each vertebra in the digital image based on a highestcorrespondence to a pre-defined mean vertebra model.
 20. The method ofclaim 19, wherein the at least one vertebra is a thoracic vertebra, andthe plurality of pre-defined mean vertebra models are twelve thoracicvertebra models.